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(54) RALLY GAME SYSTEM USING VIEW LABEL 

(57)Abstract: 

PROBLEM TO BE SOLVED: To present (X»mputer- 
derived topographic information and an image showing 
real view to each user while matching the topographic 
information with each part of the image and to make an 
automobile rally game possible In a real city. 
SOLUTION: This system transmits position Information 
(with time) about the automobiles of users and 
automobile infbnmation from a terminal 1 0 to a center 2( 
at regular time intervals. A game infonmation control part S|fM=!#|%S 
25 controls the position infbmiation about each iil^^^^^^^ 
automobile and the automobile infbmiation. and delivers pffi|ift|iiiMii 
the position infbm^on about the users and order liiP^i^iigllf 
information to a label Infomnation originating part 22. The ggif 
label information originating part 22 originates rally order L'slliSiiis^l^w^^^^^ 
label information such that rally order infonnation corresponds to an automobile area in an 
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image of the view. The rally order label information is delivered to the label information output 
part 17 of the terminal 10 and displayed on the windshield of the automobile of each user. 
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* NOTICES * 

JPO and INPIT are not responsible far any 
damages caused by the use of this translation. 

I.This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2 **** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the Inventionjrhis invention carries out the superimposed display of tiie geographical 
infomiation about each subreglon in tiie picture to an image display device to the picture which 
the user photoed using landscape image input devices, such as a camera, or relates to the 
device which carries out an audio assist etc. and is taught to a user. 
[0002] 

Pescription of tiie Prior ArtJConventionally. there were various navigation systems as a 
system which teaches a user the geographical information about the drcumference in which a 
user is present. 

r0003 IDrawing 12 i s a lineblock diagram of the navigation device indicated by JP,8-273000 A 
The position updating part 71 which will update tiie position of vehicles with reference to road 
map data if this device inputs the position data and motion data of vehicles* The data 
generating part 72 for a display which generates the road data for a display, and ttie 
background data for a display based on map data etc.. The tiiree-dimensional dynamic-image- 
data preparing part 73 which creates three-dimensional dynamic image data based on these 
data for a display, It has the storage parts store 74, and when the user of a navigation device 
sets up a priori a running path including the destination and a course place, it has a function in 
which a course can be set up, looking at the real cine mode display screen along the road 
which actually exists instead of a map screen. 

[0004]According to this device, tiie user can see the cine mode display (for example, drawing 
13) in alignment witii tiiat course, when running In accordance with the course which actually 
exists. 
[0005] 

[Problem(s) to be Solved by the lnvention]However, when using the device, and human being 
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matches actual scenery and the geographical infomnation in the world of a computer eventually 
with the naked eye, it must be recognized what the thing In actual scenery is. That is, based on 
the sign in the map with which the cine mode display of what the actual building and road in a 
user's view, and a mountain are was carried out, etc.. a naked eye is made reliance, human 
being's brain is used unconsciously, and human being has to work matching and has to 
understand. If the map in a computer and the actual scene were compared in the street corner, 
the direction has been grasped, or the mark was found and It gazed at the direction, after 
understanding the feature of the building which exists In the direction, the map was seen 
again, and he understands what the building is. 

[0006]Forthls reason, the time and effort which compares the map and real landscape on a 
computer repeatedly, and is matched at human beings has the problem of not being omissible. 
A real landscape is hard to see and cannot take correspondence easily especially in 
semidarkness, the night etc. 

[0007]There is a fault that the automobile rally game in the inside of a personal computer has 
few senses of reality, and they are unsatisfactory. 

[0008]The purpose of this Invention Is to provide the scene label use type rally gam© system 
which can do a rally game in the actual street while matching each portion in the geographical 
infomiation on a computer, and the picture (it is hereafter called a landscape image,) of a real 
landscape and teaching a user. 
[0009] 

[Means for Solving the ProblemlThis invention creates map data on a computer beforehand as 
three-dimensional data, A position in case a picture (it is henceforth called a landscape image 
in order to distinguish from a CG image) is inputted, an angle of a camera, a focal distance, 
and image size are acquired at the time of photography, Computer graphics at the time of 
viewing from a position at the time of real landscape photography, an angle of a camera, and a 
focal distance in three-dimensional map space on a computer. (It is hereafter referred to as 
CG.) Geographical information within a picture is acquired and matching Is realized by carrying 
out the superimposed display of the geographical information to a landscape image which is a 
real landscape. This geographical information is names, such as a stmcture, or attribution 
information of those in a picture, and attribution Infonnatlon means information about all 
attributes (for example, an outline, a color, etc.) about that structure. Suppose that the word 
"structure" is used in the sense of all data that has a certain geographical stmctures in the map 
DB also Including natural geographical features, such as a mountain, a river, and the sea, in 
addition to an artificial stRJcture in this specification. In acquisition of geographical information, 
it asks for a landscape image based on a camera position, a camera angle, a focal distance, 
and image size, and asks for a structure of a multiple image. A position (a grant position Is 
called hereafter) of a landscape image In which the structure should be reflected is searched 
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for, and the superimposed display of a name or attribution information of a structure is carried 
out 

[OOlOlIn order to raise further accuracy of matching with a structure in a landscape image, and 
a structure in a CG image, a structure previously gained to each subregion of a landscape 
image Is matched with pattern matching. A CG image is created based on a gained stmcture, 
and it asks for a stmcture which matched subregron in a CG image with pattem matching to 
said subregion of a landscape image, and became a basis of matched subregion. 
[0011]Here, an example of a method of creating a CG image is described. The three- 
dimensional map DB is accessed based on a camera position, a camera angle, a focal 
distance, and image size which were acquired previously, and it asks for view space in three- 
dimensional map space. It asks for a structure in view space, and three-dimensional projection 
conversion of the solid data of each structure is carried out to this surface of projection by 
using a camera screen as a surface of projection. It hides in other structures among line data 
which furthermore constitutes a projection form of each stmcture, and the hidden line removal 
of the hidden-line data is carried out using techniques, such as a normal vector method. A 
hidden line removal is carried out and area division of the CG image is carried out based on 
the remaining line data. Since the three-dimensional map DB is used, a name of a structure 
which serves as a basis of the field for every field can be matched. 

[001 2]Then, a structure name of subregion of a CG image matched with each subregion of a 
landscape image by. pattem matching is extracted. Three-dimensional proJecBon conversion of 
the position coordinate of a structure in inside of three-dimensional map sjDace is carried out to 
a previous surface of projection, and a position coordinate of a real landscape picture which 
should superimpose an extracted structure name is searched for. Label information is created 
from a position coordinate of a real landscape picture which should superimpose an extracted 
structure name. A structure name is superimposed on a landscape image which is a real 
landscape l:>ased on label information, and it displays on vision apparatus. 
[00l3]ln this invention, in order to make an automobile rally game in the still more nearly actual 
street, the following composition is taken. 

[001 4]A communication control means sends position information (with time) and car 
information about a user to a center for every fixed time from a tenminal. Here, having called rt 
car information consists of attribution information (for example, a registration number of a car, 
the vehicle name / model / color of a car, using frequency of a transceiver of automobile 
loading, etc.) about a car which each user drives. A communication control means by the side 
of a center passes position information and car information about a user to a game information 
management fool via a center control means. In a game information management tool, each 
user's position information and car information are managed, precedence information in a rally 
of each car computed from position information about a car which a user who participates in a 
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game drives, and position information and map information is managed, and position 
infonnation and precedence information are passed to a center control means. In a center 
control means, car infonr^ation is passed to a user's position information and a precedence 
infomiatlon pan at a label information preparing means. In a label information preparing 
means, based on a users position infonnation and car Infonnation under intervention, When a 
car which exists in a landscape Image is recognized and a registfation number of the car has 
been recognized from a number plate of the car, If it can presume that the registration number 
is a registration number of a car which a certain user drives from a point of position Information 
and car information, rally ranking label information which made precedence information of a 
rally con^espond to a field of a car in a landscape image will be created, and, in addition to the 
usual label information, a center control means will be passed. Here, it becomes rally ranking 
label information from attribution information including a name of an automobile slack structure 
which a user under intervention operates, and precedence Infonnation, and its position 
Information. Via a communication control part, over a label information outputting part, rally 
ranking label information superimposes a ranking number on a picture of a car, and Is 
displayed on a windshield of a user's car. 

[001 5]A scene label use type rally game system of this invention. Consist of two or more 
tenninals and centers which were carried in a car, respectively, and said each terminal. An 
image acquiring means which acquires a picture, and a position information acquiring means 
which acquires a camera position at the time of Image acquisition, A camera attribution . 
infonnation acquisition means which acquires a camera angle, a focal distance, and image 
size at the time of image acquisition, An image processing means which divides an acquired 
picture into two or more subregions, and Information about area division, said camera position, 
said camera angle, said focal distance and said image size of said picture via a 
communications network, A communication contixil means which ti-ansmits to a center, 
tiiansmits to a center car information which is position information with time on a car of the 
terminal concerned, and the atiiibution infonnation of a car via a communications network for 
every fixed time, and receives label information and rally ranking label information from said 
center, A name of a stnjcture in said label information or its attribution information is 
superimposed on a position corresponding to a grant position in a picture, Have a label 
Infonnation output means which displays a picture on which it was superimposed on vision 
apparatus, and displays said rally ranking label information on a windshield of a car. and a 
terminal-control means to conti"ol said each means, and said center, Information about area 
division of said picture, said camera position, said camera angle, said focal distance, said 
image size, position infomnati'on with time on each terminal, and car information are received 
from said each terminal via said communications networi<, A communication control means 
which transmits said label information and said rally ranking label information to said terminal. 



httpyAmw4.ipdl.mpitgo jp/cgi-bin/tran 4/1 5/2009 

PA6E12O7'RCVDATSf13/20(l911:59:36 AM [Eastern Daylip 



^^V.lMhi^^^^^^ CORP/nON] NO. 946'age t?. 1 3/3? 



A map information management tool which gains a structure which manages map infomiation, 
asks for view space in map information space based on a camera position, a camera angle, a 
focal distance, and image size which were received, and exists ail over the view space, A 
game information management tool which manages position information with time and car 
information of each terminal, and manages precedence Infomiation in a rally of each car which 
a user who participates in an automobile raQy game drives. Said label information which 
matches said gained structure to said subregion of said picture, and includes a name or 
attribution information, and a grant position of said stnjcture which were matched is created, 
Based on a user's position information and car information under rally game inten/ention, 
When a registration number of the car has been recognized from a number plate of the car and 
tine registi^tion number can presume a car which exists in a landscape image to be a 
registration number of a car which a certain user drives from a point of position information and 
car Infomiation, It has a label information preparing means which creates rally ranking label 
information which made precedence Infomriation of a rally game correspond to a field of a car 
in a landscape image, and a center control means which controls said each means. 
[OGiejAccording to the embodiment of this invention, a label information preparing means, A 
CG image which is a computer graphics image is created based on a gained structure, 
Subregion in said CG image Is matched with pattern matching to said subregion of said 
picture, it asks for a structure of matched subregion, and label information including a name or 
attribution information, and a grant position of the structure is created. 
[0017]According to other embodiments of this invention, a label information preparing means. 
Carry out three-dimensional projection conversion of the gained sbtjcture to a camera screen, 
eliminate a stixicture which does not appear from a viewpoint, and a CG image is created, A 
border line of subregion in a CG image divides a CG image into subregion, It asks for a 
stmcture which matched said subregion of said picture, and said subregion of said CG image 
with pattern matching, matched to subregion of a picture, and became a basis of subregion of 
a CG image, and label Information including a name or attribution infonmation, and a grant 
position of the stmcture is created. 
[0018] 

[Embodiment of the InventionJNext, the embodiment of this Invention is described with 
reference to drawings, 

[0019]Reference of drawing 1 w ill constitute tiie scene label use type rally game system of one 
embodiment of tills invention from tine terminal 10, tiie center 20, and tiie communications 
network 30 which were carried in ttie car. Here, two or more tenfDinals 10 exist actually, 
altiiough the deer graphic display even of tiie **** top is not carried out. 
[00201The position infonmation acquisition part 12 which acquires the image acquiring part 1 1 
from which the terminal 10 acquires a picture, and tiie camera position at One time of image 
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acquisition, and the camera attribution information acquisition part 13 wliich acquires the 
camera angle, the focal distance, and image size at the time of image acquisition. The image 
processing portion 14 which divides the acquired picture into two or more subregions, and the 
information and camera position about area division, the camera angle, the focal distance and 
image size of said picture are ti^nsmitted to the center 20 via a communications network, the 
attribution information (tiie registration number of a car.) about tfie position information with 
time on a car and tiie car 1 0 of the terminal 1 0 The car information which consists of using 
frequency of tiie transceiver of a vehicle name / model / color, and automobile loading, etc. Is 
transmitted to the center 20 via the communications network 30 for every fixed time, The 
communication conb-ol part 15 which receives label information and rally ranking label 
information from tine center 20, The label information outputting part 17 which superimposes 
the name of the structure in label information, or its attribution information on tiie position 
corresponding to the grant position In a picture, and displays the picture on which it was 
superimposed on vision apparatus, and displays rally ranking label information on the 
windshield of a car, It comprises the terminal control section 16 which controls each part Il- 
ls, and 17. 

[0021]The center 20 receives the information and camera position about the area division of a 
pichjre, a camera angle, a focal distance, image size, the position infomiation witii time on 
each temriinal, and car information from each terminal 10 via the communications network 30, 
The communication control part 23 which transmits label information and rally ranking label 
information to the tenminal 1 0. The map information Management Department 21 which gains 
the structure which manages map cnformation, asks for view space in map information space 
based on tiie camera position, the camera angle, tiie focal distance, and image size which 
were received, and exists all over the view space, The game information Management 
Department 25 which manages the position infomiation with time and car Information of eadi 
terminal 10, and manages the precedence infonnation in the rally of each car which the user 
who participates in an automobile rally game drives. The label infonnation which matches the 
sfructure gained to the subregion of said picture, and includes the name or the attribution 
information, and the grant position of a structure which were matched Is created, Based on a 
user's position Information and car informsation under rally game intervention, When the 
registration number of the car has been recognized from the number plate of the car and the 
registi-ation number can presume the car which exists in a landscape image to be a registration 
number of the car which a certain user drives from a point of position information and car 
information, It comprises tiie label information preparing part 22 which creates tiie rally ranking 
label infonnation which made the precedence information of the rally game superimpose on 
the field of tiie car in a landscape image, and tiie center control section 24 which controls each 
part 21-23, and 25. 
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[0022]Next, operation of this embodiment is explained with reference to the flow chart of 
drawing 2. 

[0023]First, in order that the terminal control section 16 may acquire the information about a 
landscape image, a processing start command is sent to the position information acquisition 
part 12, tiie camera attribution irrfomiation acquisition part 13. and the image acquiring part 11. 
The position infbnnation acquisition part 12 collects position information per second by a GPS 
receiver etc. in response to a command from the terminal control section 16, and passes it to 
the tenmlnal control section 16 (Step 21). Here, a time interval is good not only for how [ a 
second bit but]. The camera attribution information acquisition part 13 acquires the camera 
angle of landscape image recorders, such as a camera at the time of picture photography, in 
the group of a horizontal angle and an ascending vertical angle in response to the command of 
the terminal control section 16 (Step 22), and if it is a landscape image device which has a 
zoom function simultaneously. It will acquire a focal distance (Step 23). The image acquiring 
part 1 1 acquires a landscape image per second from the terminal control section 16 in 
response to a command, and passes it to the control section 16 (Step 24). Since image size is 
immobilization for every landscape image device, the terminal control section 16 holds Image 
size information. The terminal control section 16 holds the collected information as a 
landscape image file. 

[0Q24 ]Drawing 3 s hows the file format of the data structure of a landscape image file. A 
landscape Image file has header infonnation and Image data. As header information, it has 
position Information, camera angle information, a focal distance, time information, the image 
size of a graphics file, a type, and size. As position information, it has each data (for example, 
east longitude 137-degree 55 minutes and 10 seconds, north latitude 34-degree 30 seconds of 
34 minutes, altitude of 101 m 33 cm, etc.) of tiie east longitude, the north latitude, and altitude. 
As a camera angle, it has each data (for example, 254 horizontal-angle right-handed rotations, 
15 ascending vertical angles, etc.) of a horizontal angle and an ascending vertical angle. Focal 
distance data is the focal distances (for example, 28 mm etc.) of the camera lens at the time of 
picture photography. As time information, it has the time at the time of photography (for 
example, Japan Standard Time 15:06 1 7 etc. seconds etc. on January 31 , 1997). As image 
size of a graphics tile, it has pixel size (for example, 640x480 grade) in every direction. 
Similariy it has file types (TIFE form, an 8-bit color, etc.). Similariy it has the numbers of bytes 
(307.2 KB etc.) of a file. It has tiie image data itself in a binary format. 
[0025]If the terminal control section 16 stores a landscape image file, to the image processing 
portion 14, it will extract a border line from a landscape image, and will order to divide a 
larrdscape image Into two or more^fields. In the image processing portion 14, if it says roughly, 
a differential process will be performed based on tiie density difference in a landscape image, 
a border line will be extracted (Step 25), and area division is carried out by performing labeling 
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bordering on the border line (Step 26). The technical term called labeling used here rs a 
technical term used in the area division of a picture. As a procedure, a picture is first changed 
into monochrome shade image. Since an outline is a portion into which a luminosity changes 
suddenly, a border line is extracted by perfonming a differential process and asking for a 
portion with a larger differential value than a threshold. At this time, the line width of a border 
line is 1 pixel, and the border line is connected. Therefore, thinning is performed and the 
connected line with a line width of 1 pbcel is obtained. A differential process and thinning are 
enough here, if a certain technique is used from the former. 

I0026]The obtained border line is considered to be a border line of a field, and operation of 
numbering the field constituted by a border line is perfbrnied. The maximum number in the 
number turns into the number of fields, and the pixel number in a field expresses the area of 
the field. The example which divided the landscape image into two or more subregions is 
shown in drawing 8, The measure of the similarity (nearness) between fields may be 
introduced and clustering processing which summarizes two or more fields in which character 
is alike to one field may be performed. It is good also by the clustering method like the existing 
method throat. 

[0027]lf area division processing of a landscape image is completed, the communication 
control part 15 will transmit the information (header information of a landscape image file) 
about the area division of the acquired camera position, a camera angle, a focal distance, 
image size, and a picture to the center 20 with directions of the temninal control section 16 
(Step 27). These Information is passed to the map information Management Department 21 of 
the center 20 through the communication control part 23 and the center control section 24, and 
the map information Management Department 21 accesses the map DB, and performs 
calculation processing of view space (Step 28). There is a map database program as an 
example of the map information Management Department 21 . The map information 
Management Department 21 has managed three-dimensional map data. Although two- 
dimensional map data may be sufficient, since there is no height information in that case, the 
accuracy of the grant position of the labeling to a real landscape is Inferior. Height information 
is compensated and processed when carrying out based on two-dimensional map data. For 
example, if there is number-of-stories information as which a house expresses several stories 
when it is two dimensional data of a house, tiiree-dimenslonal data will be created based on 
the height infonnation which multiplied the number of stories by fixed numbers, presumed tiie 
height of the house, was presumed to be two dimensional data, and was searched for. Even 
when there is no number-of-stories information, height information can be presumed by 
assigning a fixed number of height according to tine area of a house figure, and three- 
dimensional data is simifarty created based on presumed heigiTt information. In this way, tiiree- 
dimensional data is created and processing is advanced. 
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[0028]The example of threenJimensional map data is shown in drawing 4 , The map infomiation 
space expressed by two dimensions is shown in drawing 4 (1). and the map information space 
expressed by the three dimension is shown in drawing 4 (2), In response to the command of 
the center control section 24. view space is computed based on the header information of a 
landscape image file to this three-dimensional map information space at the map Information 
Management Department 21 (Step 28). The example computation of view space is shown In 
drawing 5, First, XY axis shall stretch horizontally and the Z-axis shall stretch perpendicularly. 
From the position information in the header Information of a landscape image file, the position 
of the viewpoint E is set up in three-dimensional map Information space. For example, if it is 
east longitude 137-degree 55 minutes and 19 seconds, and north latitude 34'-degree 30 
seconds of 34 minutes and the altitude of 101 m 33 cm, the corresponding coordinates in the 
map mesh number corresponding to it will be set up. Similarly a camera angular direction is set 
up based on the horizontal angle and ascending vertical angle In the camera angle infomiation 
in header information. The focus F Is taken at the point which progressed by the focal distance 
from the viewpoint E on the straight line showing a camera angular direction. A sight line 
direction vector is a unit vector of the length 1 which comes out from the viewpoint E on the 
straight line. The width x in the X-axis of a camera screen is set up from lateral size with the 
image size of a landscape image file, and the width y in a Y-axis is set up from the size of a 
lengthwise direction. To a sight line direction vector, the flat surface of the horizontal x length y 
Is vertical to a camera angular direotion, and it Is set up so that the focus F may be included. It 
asks Ibr the straight line which connects the point of four comers of a camera screen from the 
coordinates of the viewpoint E respectively, and let three-dimensional space which four half 
lines prolonged from the viewpoint E make be view space. The example of the view space in 
three-dimensional map space is shown in drawing 6 . It looks at three-dimensional map space 
from XZ flat surfece. The portion sun-ounded with the slash in drawing 6 is a sectional view in 
XZ flat surface of the space belonging to view space. The building and mountain in view space 
are contained In the example of drawing_6. 

[0029]lt asks for the structure which exists in the view space for which it asked at the map 
information Management Department 21. Each peak which consdtutes the solid showing a 
structure calculates whether it exists in the interior area of view space for every structure. 
Usually, two-dimensional map space is divided in the twonJimensional mesh of certain size. In 
addition to the mesh of the direction of two dimensions in every direction, as how to cut the 
mesh of three-dimensional map space, a mesh is cut with the constant interval also to the 
height direction. Space will be divided in the unit space of a rectangular parallelepiped. First, 
the e)dstence of a tapped part with view space is investigated the whole unit space of a 
rectangular parallelepiped, and the number of three-dimensional unit map space with a lapped 
part is searched for. The number of three<limensional unit map space here is the same as that 
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what is called of a mesh number. Ttie existence of view space and a lapped part is 
investigated to the stnjcture in three-dimensional unit map space with a lap. It asks for the 
straight line which connects the coordinates of the peak and the coonjinates of a viewpoint 
which constitute a structure, and if the straight line has an intersection to the camera screen of 
drawing 7. it is in view space. If at least the one peak fulfills this condition among two or more 
peaks which constitute a structure, that structure shall have view space and a lapped part. 
[0030]When a structure is contained in the inside of view space or that part is included, the 
processing which cames out tiireeKlimensional projection conversion of each structure to this 
surface of projection is started by using a camera screen as a surface of projection (Step 29). 
Here, as shown in drawin^^ . after reexpressing the point P by the coordinate system based 
on the viewpoint E based on a following formula (1), the point P is projected on a camera 
screen and It asks for the intersection Q. 
[0031] 
[Equation 1] 




Coordinate-vectors L- of the coordinate-point E=(ex, ey, ez):viewpoint of the peak which 
constitutes a point P=(x, y, 2):structure here (Ix, ly, \z): Sight line direction vector (unit vector) 
Coordinates r=(lx^+ly^) '^^^ intersection Q= at the time of expressing by the coordinate system 
based on the viewpoint E of the point P=(x', /, 2'):point P PC Y): The projected point t to the 
camera screen of the point P is a focal distance. tOQ321In three-dimensional projection 
conversion, the fieM which tiie peak stretches for every structure first is searched for. For 
example, six fields can be found if it is a structure expressed in a rectangular parallelepiped. 
When carrying out projection conversion of each field to a camera screen, to each pixel on the 
camera screen Included in a projection area, the distance of a viewpoint and the corresponding 
points on the field is calculated, and it stores in a memory as a depth value (Z value). For 
every field of each structure, the deptii value (Z value) over each pixel on a camera screen Is 
calculated, and it stores in a memory, z' in one expresses tiie depth value (Z value) from a 
vievirpoint (formula). 

[0033]There are a structure which appears from a viewpoint, and a structure not appearing in 
Inside of a structure by which three-dimensional projection conversion was earned out to a 
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camera screen. It is necessary to ask only for a structure which appears from a viewpoint in It 
and to search for a field located from a viewpoint In an opposite hand, and a field rnlermpted 
by other structures. Then, hidden surface elimination is performed (Step 30). Although It is in a 
method of hidden surface processing variously, a Z buffer algorithm is used, for example. 
Other scanline algorithms and the ray tracing method may be sufficient. 
f0034]A field which takes a pixel on a camera screen arbitrarily and takes the smallest depth 
value to the pixel is searched for. Thus. If sequential opeiation is continued about each field of 
each structure, a field nearest to a viewpoint will be left behind for every pfacel on a camera 
screen. Since a camera screen top pixel in which a field nearest to a viewpoint Is determined 
for every pixel on a camera screen, and a field nearest to a viewpoint is common generally 
constitutes a field, in a camera screen, two or more fields which consist of a pixel which makes 
a common field the nearest field are made. In this way. a field which was able to be found is a 
field of a result of having carried out three-dimensional projection conversion of the subregion 
of a structure which appears from a viewpoint A field in an opposite hand and a field 
interrupted by other structures are eliminated .from a viewpoint 
[OOSSJIn this way, a made field fonms a CG picture area (Step 31). 
[0036]To apex coorelinates of a two-dimensional figure which constitutes a CG picture area, 
three-dimensional coordinates before projection conversion are searched for, and it stores in a 
memory by making both correspondence relation into link infomiatlon. It Is used for asking etc. 
of which structure the tw&.dimenslonal field is a projection based on link information. 
[00371A hidden line removal Is canied out and area division of the CG image is carried out 
based on the remaining line data. Since the three-dimensional map DB Is used, a name of a 
sfructure which serves as a basis of the field ft>r every field can be matched. A field to which a 
CG image was divided is numbered in order. An example which divided a CG image into two 
or more subregions Is shown in drawing 9 . 

[0038]lf area division processing of a CG image is completed, the center control section 24 will 
order to the label information preparing part 22 to match a region division of a CG image, and 
a region division of a landscape Image. In the label information preparing part 22, a region 
division of a CG Image and a region dMskjn of a landscape Image are matched by template 
matching (refer to Step 32 and drawing 10) . . 

[0Q39JIt matches with a region division of a CG image sequentially from a young field (for 
example. No. 1) of a number among region divisions of a landscape image. In matching, 
although it is very good in which [ of the former to a certain MACHINGU methods ], the simple 
template-matching method is taken here. That is. when you pile up two fields to compare and 
there is a ratio of an overlapping portion more than a fixed ratio fixed as a threshold, suppose 
that it matches as a field about the same structure. For example, a coordinate value of each 
pixel in the field is set to (A, B) about R1 of the 1st region division of a landscape image. 
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Values of a pixel in coordinates (A, B) are an inside of afield, therefore 1. In the 1st region 
division SI of a CG image, if coordinates (A. B) become In the field Si. it will be the pixel value 
1 and will lap, but if it becomes outside S1, it will be the pixel value 0 and will not lap. In this 
way. as the lap coefficient K (A, B) in coordinates (A, B), 1 and when lapping, and not lapping, 
it Is dedded by 0. Coordinates (A. B) are moved in the field R1, and the lap coefficient K (A. B) 
is calculated. And to several N1 of coordinates (A, B) moved in the field R1, the lap coefficient 
K (A, B) calculates several N2 of coordinates which were 1, and a thing corresponding to a 
case where N1/N2 are more than a threshold, in the region division R1 of a landscape Image 
and the region division SI of a CG image is decided on. This matching is performed from the 
1st of a region division of a landscape image to the last thing. Even if a position gap of some is 
in an XY direction in addition to this as a matching method, a valuation function which 
becomes the same value may be used. 

[0040]After matching subreglon of a CG image to subregion of a landscape image in the label 
infonnaton preparing part 22, It is matehed, a structure of subregion is extracted (Step 33). 
and processing (Step 34) which searches for infomfiatlon (a name or attribution Infbrrnatlon of a 
stnjcture) which should be further superimposed for every subregion of a landscape image, 
and is created as label information with a position which should be superimposed is started. 
First subregion of a corresponding CG image is taken out to subregion of a landscape image. 
Subregion of a taken-out CG image is obtained by carrying out three-dimensional projection 
conversion of the field which has a three-dimensional structure thing in three-dimensional map 
space from the first to a camera screen. Then, a depth value (Z value) In which subregion of a 
CG image has a field of a three-dimensional structure thing used as a baste of three- 
dimensional projection conversion is calculated as a key. When three-dimensional projection 
conversion is carried out previously, created link infomfiation may be used as a key. Based on 
a field of a structure used as a basis, the three-dimensional map DB is accessed and a name 
or attribution information of the structure is acquired. It is [ anything ] good if attribution 
information is information which means information which accompanies about the structure 
and starts the structure here. And a position coordinate which should superimpose a name or 
attribution infennation is decided to subregion'of a landscape image. How to detennine may 
determine how. For example, the center of figure which sbietches subregion may be sufficient 
Label information is created from a name or attribution information, and a grant position 
coordinate of the structure. An example of label infomiation is shown in Table 1 . 
[0041] 
Uable 1] 
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The label information preparing part 22 will pass label informafion to the center control section 
24, if it finishes creating label infomiation. The center control section 24 will order to transmit 
label information to the tenmrnal 10 to the communications control 23, if label information is 
received. (Step 35). 

[0042]LabeI information is passed to the label information outputting part 17 of the tenninal 10, 
and the label infomiation outputting part 17 superimposes a name or attribution information of 
a structure in label infomiation on a position in a landscape Image (Step 36), A landscape 
image on which it was superimposed is displayed on graphic display devices, such as a 
display (CRT) and a head IWARAN toddy spray (Step 37). An example of a landscape image in 
which it was superimposed on label information is shown in drawing 11 , 
[0043]lf an automobile rally game is started between a user of the tenninal 10, and a user of 
other temiinals, the communication control part 15 will send position information with time and 
car information of a car which a user of tiie terminal 10 operates to the center 20 for every 
fixed time witii position infonr^ation with time on the terminal 10 (Step 38). Through the center 
control section 24, position infomiation with time and car infomnation of a car of the tenninal 10 
are passed to the game infonnation Management Department 25 of the center 20, and the 
game information Management Department 25, Each user's position information with time and 
car Information are managed, precedence infonnation in a rally of each car which a user who 
participates in an automobile rally game drives Is managed, and a user's position information 
and car infonnation are passed to the center control section 24 (Step 39). A user's position 
information and car information are passed to tiie label information preparing part 22 by the 
center control section 24. and in the label information preparing part 22* When a registration 
number of the car has been recognized from a number plate of the car, a car which exists in a 
landscape image based on a user's position information and car Information under rally game 
Inten^ention, If ttie registration number can presume a registi^tion number of a car which a 
certain user drives from a point of position infonnation and car information, rally ranking label 
Information on which a field of a car in a landscape image was made to superimpose 
precedence information of a rally game will be created, and the center conti-ol section 24 will 
be passed (Step 40). 
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[0044]The center control section 24 transmits rally ranking label Information to the terminal 10 
(Step 41). The label information outputting part 17 of the terminal 10 displays rally ranking 
label information on the windshield of a car (Step 42). 
[0045] 

[Effect of the InventionlSince according to this invention the geographical information on a 
computer and each portion in the landscape image of a real landscape can be matched and a 
user can be shown, as explained above. Human being compares the map and real landscape 
on a computer, and ft does not match at human beings, ** also ends, and an automobile rally 
game is made in the actual street, and the existence and ranking understand only a rally . 
participating vehicle in a national highway etc. 



{Translation done.] 
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